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SUtflARY 


One of the principal objectives of the NASA AgRISTARS program Is the 
Inventory of global crop resources using remotely sensed data gathered by 
Land Satellites (Landsat). a central problem In any such crop Inventory 
procedure Is the Interpretation of Landsat Images and Identification of 
parts of each Image which are covered by a particular crop of Interest. 

This task of "labeling" Is largely a manual one done by trained human ana- 
lysts and consequently presents obstacles to the development of totally 
automated crop Inventory systems. However, development In Knowledge 
Engineering as well as widespread availability of Inexpensive hardware and 
software for Artificial Intelligence work offers possibilities for develop- 
ing expert systems for labeling of crops. Such a knowledge based approach 
to labeling is presented in this paper. 


INTRODUCTION 


The Landsat spans different parts of the earth's surface providing 
images at regular periodic Intervals. Images gathered by Landsat are In 
the’form of picture elements (or pixels) consisting of the average spectral 
response of the area cc ered by each pixel (about an acre) in four visible 
and near infrared frequency bands. It is known however that it is possible 
to make a transformation of this four-channel data onto a two-dimensional 
plane essentially preserving the Information contained In the original data. 
This permits creation of visual displays to be used by analyst Interpreters 
(labelers) for assigning labels to different parts of the scene, (cf: 

Ref. [2], [4], and [6]). It Is to be noted that labeling forms only a part 
of the overall remote sensing crop Inventory exercise, but as experience 
has shown, a most crucial part. It may also be mentioned that labeling of 
images arises in other applications besides crop Inventory by remote sensing 
(Ref. [9]). 

In all decision making processes Including labeling, human experts 
use knowledge which Is not easy to formalize. Such expert knowledge can 
frequently consist of neurlstic rules, elimination by constraints, exercise 
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of judgment, all gained through experience. In the context of labeling, 
experienced analysts make successful use of spatial context and texture. 

It is precisely tnese skills which one would like to impart to an automatic 
labeler. 

One of the r'.st intriguing concepts in automation technologies is the 
idea of employing Artificial Intelligence (AI) systems in a decision maxing 
capacity, in effect replace human experts by "Expert Systems." One of the 
most active areas of AI research is the application of such techniques to 
problems in Pattern Recognition/ Image Understanding (Ref. [8] and [9]). 

This development is hardly a coincidence. AI and Pattern Recognition 
share several common features among which the most significant is that they 
are "knowledge" based (Ref. [8]). A fundamental requirement in AI approaches 
is that one be ab1«> to represent and process "knowledge" and not merely data. 
We briefly discuss a knowledge based approach to labeling in remainder of 
this paper. 


KNOWLEDGE BASED APPROACHES 


Knowledge Engineering is the branch of Artificial Intelligence used in 
Dull ding expert systems. In it one attempts to capture the essential 
problem solving skills of an expert, transmit those same skills to a com- 
puting system thereby creating an automated expert or an expert system. 

It is now recognized (Ref. [2]) that human problem solveis possess know- 
ledge and techniques which are specific to a problem area and not general 
problem solving skills. Furthermore, this knowledge is frequently heuristic 
knowledge consisting of judgment, experience, good practice, and so on. 

Thus, it is clear that any expert system has to include a store of knowledge 
called "Knowledge Base" and a set of techniques usually called "Paradigms." 
Consequently, an effective and flexible representation of knowledge is a 
crucial first step in attempting to develop any expert system. 

The pioneering effort in knowledge based problem solving was undoubtedly 
Slagle's integration program (Ref. [8]). Since that time many expert problem 
solvers ranging in applications from advanced mathematics to medical diag- 
nosis have been st'ccessfully built. Of these we can cite MACSYMA (advanced 
mathematics), MYCIN (diagnosis of blood disorders), and DENDRAL (chemical 
analysis of spectroscopic data) as examples of highly successful knowledge 
based expert systems. As a result of these efforts, the following general 
insights have been gained. 

1. Any Artificial Intelligence work entails storage and manipulation 
of complex data structures. 

2. These complex data structures which represent "knowledge" can 
include data as well as programs. Hence, the programming environ- 
ment must facilitate uniform representation of data and programs. 


- 684 - 


3. This activity involves "knowledge processing" and not just "data 
processing." 

4. Such a knowledge based system should permit ease of frequent 
modification as we update our knowledge, without dismantling the 
system. This is known technically as "embedability." 

5. These conditions argue for a LISP base for AI systems. In fact 
LISP is the sine qua non for serious AI work. (Ref. [8]) 

In addition to its uniform treatment of data and programs as symbolic 
expressions, LISP as a prograimilng language provides several extremely 
powerful features such as laiflbdas and recursive function calls for transfer 
of control and use of programs and functions as arguments for other programs. 

We can represent a knowledge base by the following general schematic 
diagram. 
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KN0WLErM5E BASE FOR THE EXPERT LABELING SYSTEM 


Recall that we are Interested In assigning different parts of a scene 
to different land cover types based on Its Landsat Imagery or "label" the 
scene. We also noted that the 4-d1mens1onal data obtained by the multi - 
spectral scanner OiSS} are transformed Into a two-dimensional space known 
as the "greenness-brightness" space. Consequently, data to be used for 
labeling will consist of multitempural greenness -brightness values for each 
pixel. We shall see later that there are certain minimal data requirements. 
The production rules will be based on a study conducted by Palmer and Magness 
(Ref. [5]), and contain a very large heuristic base. These riles will be 
displayed in a widely used version of LISP known as "MACLISP" (Ref. [9]). 

Ue shall not discuss the labeling paradigms separately but merely note that 
Inference rules with tie help of IF THEN — rules by COND constructs pre- 
sent no particular difficulties (Ref. [9]). 

Because of atmospheric conditions not all the Landsat passes over a 
scene result In usable data acquisitions. Such deficiencies result 1n 
Imperfect or Incomplete labeling. For the sake of exposition, we assume 
that we are attempting a two-stage decision, assigning a label as Spring 
Small grains or not at stage 1 and as barley or not at the final stage. 

In order for a pixel to be labelable, the minimal acquislon requirements 
are expressed In terms four "windows" which are time Intervals In which 
data are available. Based on Palmer-Magness study, we define the four 
"windows" which are Intervals represented by lists as follows: 

(SETQ W1 (LIST 

(DIFFERENCE XI 5) (PLUS XI 18))) 

Where XI Is computed from the crop ca^^ndar time when at least 50 percent 
has been planted for wheat. Based . similar considerations, windows W» 
for wheat W^ and W. for barley arc .iefined. These values for XI, et ar 
are determined from models for crop calendars. Essentially they ascertain 
availability of observations during certain crucial crop development stages. 
It Is an Interesting empirical observation that minimum data requirements 
are different for different geographical regions. This "contextual Informa- 
tion" forms a crucial part of the knowledge base. Next, we check to see 
If the Landsat data are adequate for labeling. For each pixel, there Is a 
list of acquisition dates from which we decide whether or not the region 
of that pixel Is labelable. The following function checks If the list 
DATA contains an acquisition In particular window by recursive scanning 
the list DATA. 


(COND ((NULL DATA) 'ABSENT) 
((GREATERP (CAR WINDOW) 
DATA) 
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CCAOR WINDOW)) 'PRESENT) 
CT (CHECK WINDOW (CDR DATA))))) 


For example, the function (CHECK W1 DATA) will return PRESENT or ABSENT 
whether or not DATA contains an acquisition In the window Wl. This function 
Is evaluated for each of the four windows. Then It Is a simple matter to 
write LISP function wttich determines whether the minimal data requirements 
are met, for that particular geographical region. In addition, the knowledge 
base Includes Information concerning quality of the data In the form of cloud 
cover, haze, etc. If such cover exceeds 40 percent, the '.cene cannot be 
labeled. The actual l'‘.bel1ng procedure or “Paradigm" uf.s a scatter plot 
generated from windows 2 and 3 and a decision line which will not be dis- 
cussed here. It Is to be noted that we have given only the outline of the 
procedure omitting most of the details. An alternative approach, based on 
an angular statistical measure to crop classification can be found In Ref. [7]. 

So the knowledge base for an expert system based on the scheme described 
above can be envisaged to be built around the following skeleton. 

1. An eliminating constraint which discards data with excessive haze 
and/or cloud cover. 

2. A production rule which determines the four windows as described 
earlier. (The window function.) 

3. A decision rule which checks the geographical location of the scene 
against the minimum data requirements In the form of acquisitions 
in the windows. (The function CHECK.) 

4. Data will comprise of crop calendar information as well as Landsat 
data. 

It is interesting to note with the exception of the data, all the rule 
given above use programs or functions as components in the knowledge base. 
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